Process for coding image sequences in a transmitter unit

ABSTRACT

The method achieves a reduction of the required transmission capacity. This occurs in that pixels of pixel-based image parts (PB) and corresponding pixels of computer image parts (CB) that overlap are already merged in the transmission unit according to the blue part of the pixel of the pixel-based image part (PB) and according to the computer model attributes, so that a key signal (K) is no longer transmitted. The key signal (K) usually indicates the ratio in which the pixel of the pixel-based image part (PB) should be merged with the pixel of the computer image part (CB). In known methods, this information is not binary. In the receiver unit, thus, the only thing decided is whether the pixel is taken from a partially mixed image sequence (TGB) or from the computer image parts (CB).

BACKGROUND OF THE INVENTION

In the encoding of image sequences that represent a combination ofcomputer image parts, for example image parts that are constructed fromcomputer graphics, and pixel-based image parts, for example image partsthat are registered by a camera, it is known to separately encode andtransmit the input data of the computer graphics (script files withassociated data) and the pixel-based image parts. It is only at thereceiver that the graphics models are then rendered and mixed with thepixel-based image parts. Since the mixing is not a binary operation, theinformation about the ratio in which the mixing should be carried outmust be additionally transmitted for each pixel. This information isstored in a key signal and is acquired by a color segmenting from thecamera signal in traditional television (F. Fechter et al., "Signalprocessing for a digital HDTV chromakey mixer", Signal Processing: ImageCommunications 5, 1993, pp. 417-423 and T. Sprague, "Merging DigitalTechnology into an Analog World", SMPTE Journal, February 1994, Vol.103, No. 2, 1994, pp. 100-104).

Methods are known for transmission that separately encode computer imageparts and pixel-based image parts in a transmission unit, together withwhat is referred to as a key signal that indicates the ratio in which apixel of a computer image part is to be respectively mixed with thecorresponding pixel of the pixel-based image part. In blue boxexposures, for example, the key signal is acquired from the pixel-basedimage parts by a color segmenting. The intensity of the blue colorationthereby indicates the later mixing ratio in which the pixel informationof the pixel of the computer image part is mixed with the correspondingpixel information of the pixel-based image part in the receiver unit.The mixing of the two pixel information in the receiver unit is not abinary mixing.

Since the mixing is not binary, much information for the respectivelymixing ratio must be transmitted for every dot with the assistance ofthe key signal. The key signal thus requires a considerable amount oftransmission capacity. However, the information of the key signal has noadditional informational content for the image itself but merely theinformation about the ratio in which the images should be mixed togetherin the receiver unit.

SUMMARY OF THE INVENTION

The invention is based on the problem of already partially mixing imagesequences composed of computer image parts and pixel-based image partsin a suitable way and encoding them and thereby eliminating thetransmission of the key signal that is normally additionally required.

In general terms the present invention is a method for encoding anddecoding image sequences using computer image parts and pixel-basedimage parts on the basis of a key signal. In a transmission unit, pixelinformation of a pixel of the computer image part is mixed dependent onthe key signal with pixel information of a corresponding pixel of thepixel-based image part to form a partially mixed image sequence. Thepartially mixed image sequence and the computer image part aretransmitted from the transmission unit to a receiver unit. Onlycompletely covering regions of the computer image part are rendered inthe receiver unit. The completely covering regions of the computer imagepart are inserted into predetermined regions of the partially mixedimage sequence.

Advantageous developments of the present invention are as follows.

Regions of the partially mixed image sequence that are completelycovered by pixels of the computer image parts are blanked out before theencoding.

Regions in the partially mixed image sequence having the block size ofthe encoder that are completely covered by the computer image parts havea constant luminance value assigned to them.

The inventive method discloses a possibility of modifying thepixel-based image parts of image sequences dependent on the key signalbefore the encoding in order to be subsequently able to efficientlycompress them with block-based encoders (for example, MPEC1 or MPEC2,etc.) and in order to be able to simply mix them binarily in thereceiver unit. As a result thereof, the key signal, which contains theinformation of the mixing ratio of the computer image parts with thepixel-based image parts, can be substantially foregone.

Compared to known methods, the inventive method particularly has theadvantage that the key signal that requires considerable transmissioncapacity in known methods is reduced and, thus, the requiredtransmission capacity is reduced.

A development of the method has the advantage that the image sequencecan be efficiently encoded given employment of block-based encoders (forexample, MPEG1, MPEG2, etc.).

The receiver unit can be fashioned significantly simpler in that the twoimage parts can be binarily mixed in the receiver unit dependent on thekey signal.

BRIEF DESCRIPTION OF THE DRAWING

The features of the present invention which are believed to be novel,are set forth with particularity in the appended claims. The invention,together with further objects and advantages, may best be understood byreference to the following description taken in conjunction with theaccompanying drawing, in which:

The single FIGURE shows a block circuit diagram that illustrates theinventive method.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

In what are referred to as blue box exposures, a part of the color blueof a pixel (blue part) defines how a pixel-based image part PB is to bemixed with a computer image part CB. What is to be understood in thiscontext by the computer image part CB is, for example, a computergraphic. A decision as to whether and to what extent a correspondingpixel of the computer image part CB is to be inserted is made on thebasis of the pure blue part of the respective pixel in the pixel-basedimage part PB. The intensity of the blue coloration thereby indicatesthe later mixing ratio of the pixel information of the pixel of thepixel-based image part PB and of the corresponding pixel of the computerimage part CB. When a pixel of the pixel-based image part PB is notcolored blue, then it remains unaffected, i.e. the pixel of thepixel-based image part PB is not merged with the corresponding pixel ofthe computer image part CB.

In computer models, whether areas of the computer model are transparentis additionally indicated for the computer image part CB by the materialproperties. The degree of transparency is likewise stored as computermodel attribute. The coordinates in which the computer models are to beinserted into the pixel-based image part PB are also indicated. Thecalculation of illumination effects (shadow, mirroring, etc.) ensuesseparately. As a result thereof, a decision as to which pixels of theimage are to be darkened or lightened can be made in what is referred toas the rendering. This information respectively enters into therespectively affected pixel, so that this information also enters in inthe merging of the pixels of the pixel-based image part PB and thecorresponding pixel of the computer image part CB.

In the calculation of a key signal K, all those areas that must bepresented transparent, shadowed or lightened in the rendering of thecomputer models are also additionally written, first, into the keysignal K in the form of pixel areas. The degree of the transparency aswell as, potentially, illumination effects are co-indicated, these beingencoded as gray scale values. Subsequently, those pixels that arecolored blue in the pixel-based image part PB are suitably added to thekey signal K according to their blue intensity.

The key signal K, with whose assistance the transmission unit can easilydecide whether a given pixel must be mixed or not, is obtained on thebasis of the procedure described above. For example, the decision in thetransmission unit can be made according to the following rules: when thecorresponding pixel in the key signal K is white, then the correspondingpixel from the computer image part CB is inserted into a partially mixedimage sequence TGB.

In this context, the partially mixed image sequence TGB refers to animage sequence with images whose pixel information was merged with thepixels from the computer image part CB and the corresponding pixels fromthe pixel-based image part PB.

When the corresponding pixel in the key signal K is black, then thecorresponding pixel from the pixel-based image part PB is inserted intothe partially mixed image sequence TGB.

When the corresponding pixel in the key signal K exhibits a differentbrightness for the respective pixel, then the color information of thepixel from the information contained in the computer image part CB andthe color information of the corresponding pixel from the pixel-basedimage part are mixed together. This situation arises, for example, givenshadows of objects of the computer image part CB onto regions of thepixel-based image part, as a result whereof the color information of thepixels of the region onto which the shadow of the respective object"falls" is modified.

The mixing ratio is indicated in the key signal K and is calculated inthe transmission unit in a way familiar to a person skilled in the art.

After the partially mixed image sequence TGB was formed in theabove-described way, regions that are completely covered by regions ofthe computer image part CB are blanked out before the encoding of thepartially mixed image sequence TGB. The information as to whether aregion of the computer image part CB completely covers a region of thepartially mixed image sequence TGB is contained in an attribute of the3D models of the computer image part CB.

Subsequently, the symbolic description of the computer image part CB,i.e. a script file and associated data, and the partially mixed imagesequence TGB are encoded and transmitted to a receiver unit.

In the receiver unit, only the completely covering regions of thecomputer image part CB are rendered and inserted into the partiallymixed image sequence TGB, as indicated in the attributes of the computerimage part, for example the location of the introduced object of is thecomputer image part CB within the partially mixed image sequence TGB.

A further calculation of, for example, shadows of the introduced objectsis no longer necessary in the receiver unit since these were alreadyimplemented in the transmission unit. The only thing still remaining,thus, in the receiver unit is a purely binary mixing. The transmissionof the key signal K is eliminated.

Given employment of block-based encoders (for example, MPEG1, MPEG2,etc.), it is advantageous to set regions in the partially mixed imagesequence TGB having the block size of the encoder that are completelycovered by the pixels of the computer image part CB to a constant value.The encoding given block-based encoders is thus considerably moreefficiently configured.

A potentially provided post-processing of the image sequences, forexample an anti-aliasing for edge smoothing of the decoded, remainingcomputer image sequences CBR, is implemented at the receiver side.

The invention is not limited to the particular details of the methoddepicted and other modifications and applications are contemplated.Certain other changes may be made in the above described method withoutdeparting from the true spirit and scope of the invention hereininvolved. It is intended, therefore, that the subject matter in theabove depiction shall be interpreted as illustrative and not in alimiting sense.

What is claimed is:
 1. A method for encoding and decoding imagesequences upon employment of computer image parts and pixel-based imageparts based on a key signal, comprising the steps of:mixing, in atransmission unit, pixel information of a pixel of a computer image partdependent on a key signal with pixel information of a correspondingpixel of a pixel-based image part to form a partially mixed imagesequence; transmitting the partially mixed image sequence and thecomputer image part are transmitted from the transmission unit to areceiver unit; rendering only completely covering regions of thecomputer image part in the receiver unit; and inserting the completelycovering regions of the computer image part into predetermined regionsof the partially mixed image sequence.
 2. The method according to claim1, wherein regions of the partially mixed image sequence that arecompletely covered by pixels of the computer image parts are blanked outbefore encoding the image sequences.
 3. The method according to claim 1,wherein regions in the partially mixed image sequence having a blocksize of an encoder that are completely covered by the computer imageparts have a constant luminance value assigned thereto.